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ABSTRACT 


An Artificial Neural Network (ANN) is an information processing model that is encouraged by the way biological nervous systems, such as the 
brain, process information. The key element of this paradigm is the novel structure of the information processing system. It is composed of a 
large number of highly interconnected processing elements (neurons) working in unison to solve specific problems. ANNs, like people, learn by 
example. An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process. Learning 
in biological systems involves adjustments to the synaptic connections that exist between the neurons. This is true of ANNs as well. 
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1. INTRODUCTION 


Artificial neural networks are computational models enthused by animal central nervous systems (particularly the 
brain) that are capable of machine learning and pattern recognition. They are generally presented as systems of 
interconnected "neurons" that can compute outputs from inputs by feeding information through the network. For 
example, in a neural network for handwriting recognition, a set of input neurons may be activated by the pixels of an 
input image representing a letter or digit. The activations of these neurons are then passed on, weighted and 
transformed by some function determined by the network's designer, to other neurons, etc., until finally an output 
neuron is activated that determines which character was read. Like other machine learning methods, neural networks 
have been used to solve a wide variety of tasks that are hard to solve using ordinary rule-based programming, 
including computer vision and speech recognition. 


2. LEARNING ALGORITHMS 
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For Example: If two nodes x and y are often simultaneously active, Hebbian Learning will increase the connection 
strength between the two so that excitation of either one tends to cause excitation of the other. While, if nodes x and z 
were of opposite activations at all times, then Hebbian learning would gradually decrease the connection in between 
below zero so that an excited x and z would inhibit the other. 

For this rule the learning signal is equal simple to the neuron’s output. 

We have 

r~ =f x) 

The increment AME = c f (WE x)x 


The single weight adjustment using the following increment: 
AWE; = cf (WE x)x; 


This can be written as 
AWi;= c a; &; for j= 1, 2, 3....n 
Hebbian Learning Rule has four Features: 
e It is unsupervised. 
e It is a local learning rule which means that it can be applied to a network in parallel. 
e It is simple and therefore requires very little computation. 
e It is biologically plausible. 


Perceptron learning rule 
The Perceptron learning rule was originally developed by Frank Rosenblatt in the late1950s.Training patterns are 
presented to the network's inputs; the output is computed. Then the connection weights ™; are modified by an amount 
that is proportional to the product of 

e The difference between the actual output, y, and the desired output, d, and 

e The input pattern, x. 


The algorithm is as follows: 
e Initialize the weights and threshold to small random numbers. 
e Present a vector x to the neuron inputs and calculate the output. 
e Update the weights according to: 


wj(t+4)= w;(t+1)+9(d—y) 


where 
1.d is the desired output, 
2.t is the iteration number, and 
3.eta is the gain or step size, where 0.0 <n< 1.0 


e Repeat steps 2 and 3 until: 
1.the iteration error is less than a user-specified error threshold or 
2.a predetermined number of iterations have been completed. 


Notice that learning only occurs when an error is made, otherwise the weights are left unchanged. 

This rule is thus a modified form of Hebb learning. 

During training, it is often useful to measure the performance of the network as it attempts to find the optimal weight 
set. A common error measure or cost function used is sum-squared error. It is computed over all of the input 
vector/output vector pairs in the training set and is given by the equation below: 


E= > Ensily'— a2 


Where p is the number of input/output vector pairs in the training set. 


Widrow- Hoff Learning Rule 

The Widrow-Hoff Learning Rule is applicable for 
1.The supervised training of neural networks. 
2.Weights are initialized at any value in this method. 
3.This rule is also called LMS (least mean square). 
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4.It is independent of the activation function of neurons used since it minimizes the squared error between the 
desired output valued, and the neuron’s activation value "=t; =W= x. 


The learning signal r for this rule is 


r=d,—Wix 
AW; = c (d - Wix)a 


For single weight adjustment 


AW = c (d; — WEx)x; 


Note: Widrow- Hoff learning rule is a special case of Delta learning rule. 


If 
fiwix)= Wix 


Then 
f(met)= met and f= =4 
where net = WW x 


Then from Delta learning Rule, the learning signal is 
r=([d;— fiwtx)] f'n x) 


r=[d;— flWix)] <1 Or r=[d;-—(Wix)] 


Therefore the learning signals of delta learning rule and Widrow Hoff learning rules are equal for this special case. 


Correlation Learning Rule 


1.Correlation learning rule is for supervised learning. 
2.By substituting r = d; in general learning rule : 


w,{t)= cr [w;(t), x(t) d.(e)a[e) 


We obtain the correlation learning rule. The Weight adjustment are 
AW; = c (d;)x 


Or for single weight adjustment 
AW; = c (4; )3; 


This rule states that if d; is the desired response due to %; the corresponding weight increase is proportional to their 


product.. 


1.The rule typically applies to recording data in memory networks with binary response neurons. 
This rule is a special case of Hebbian Learning Rule with a binary activation and for @; =d;. But the main difference is 


that Hebbian Rule is performed in unsupervised mode and correlation learning rule in supervised learning mode. 
Similar to Hebbian Learning Rule, this learning rule also requires weight initialization w = 0. 
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